IOfflineNumberCommitTo3E.Run

1/15/2025 6:50:04 PM (+32.613s) Failed

An exception occurred during performance of the job.

System.Net.Http.HttpRequestException

Request failed with status code InternalServerError

System.Net.Http.HttpRequestException: Request failed with status code InternalServerError
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 167
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+30ms Processing

Server:: dw0sdwk000723:3256
Worker:: a1db6549

+33msOfflineNumberCommitTo3ECommandHandler - Started
+3.440sNo of records retrieved : 1
+3.448s----------------------------
+3.454sSubmission started for Number: 621335
+3.723sSubmission to 3E - Started at 1/15/2025 6:49:35 PM by Sam Keegan
+3.847sCurrent item:
+3.856sProject Id: 2128221, Project Number: 313716.0003, Number Id - 621335
+3.895sTrigger Type - FN, Tax Year - 2023, State/Fed - FED
+3.901sPreparing request for submission
+3.908sCommitting number to 3E
+32.454sAn error occured: Request failed with status code InternalServerError
+32.484sAn error occured: Request failed with status code InternalServerError

+1h 50m Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+3ms Scheduled

Retry attempt 10 of 10: The request was canceled due to the configured Ht…

Enqueue at:: 1/15/2025 6:49:17 PM

+1m 41.867s Failed

An exception occurred during performance of the job.

System.Threading.Tasks.TaskCanceledException

The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 167
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+20ms Processing

Server:: dw0sdwk000723:3256
Worker:: 4f8e94a8

+29msOfflineNumberCommitTo3ECommandHandler - Started
+1.089sNo of records retrieved : 1
+1.096s----------------------------
+1.102sSubmission started for Number: 621335
+1.493sSubmission to 3E - Started at 1/15/2025 4:57:10 PM by Sam Keegan
+1.581sCurrent item:
+1.590sProject Id: 2128221, Project Number: 313716.0003, Number Id - 621335
+1.612sTrigger Type - FN, Tax Year - 2023, State/Fed - FED
+1.619sPreparing request for submission
+1.625sCommitting number to 3E
+1m 41.813sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
+1m 41.822sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

+1h 10m Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+16ms Scheduled

Retry attempt 9 of 10: The request was canceled due to the configured Ht…

Enqueue at:: 1/15/2025 4:57:03 PM

+1m 42.957s Failed

An exception occurred during performance of the job.

System.Threading.Tasks.TaskCanceledException

The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 167
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+77ms Processing

Server:: dw0sdwk000723:3256
Worker:: 1f68d73a

+208msOfflineNumberCommitTo3ECommandHandler - Started
+509msNo of records retrieved : 1
+521ms----------------------------
+539msSubmission started for Number: 621335
+817msSubmission to 3E - Started at 1/15/2025 3:44:35 PM by Sam Keegan
+927msCurrent item:
+935msProject Id: 2128221, Project Number: 313716.0003, Number Id - 621335
+968msTrigger Type - FN, Tax Year - 2023, State/Fed - FED
+984msPreparing request for submission
+1.014sCommitting number to 3E
+1m 42.684sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
+1m 42.703sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

+42m 49.276s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+10ms Scheduled

Retry attempt 8 of 10: The request was canceled due to the configured Ht…

Enqueue at:: 1/15/2025 3:44:33 PM

+1m 43.970s Failed

An exception occurred during performance of the job.

System.Threading.Tasks.TaskCanceledException

The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 165
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+40ms Processing

Server:: dw0sdwk000723:3256
Worker:: 0fb14427

+69msOfflineNumberCommitTo3ECommandHandler - Started
+2.114sNo of records retrieved : 1
+2.130s----------------------------
+2.140sSubmission started for Number: 621335
+3.545sSubmission to 3E - Started at 1/15/2025 3:00:04 PM by Sam Keegan
+3.729sCurrent item:
+3.746sProject Id: 2128221, Project Number: 313716.0003, Number Id - 621335
+3.813sTrigger Type - FN, Tax Year - 2023, State/Fed - FED
+3.830sPreparing request for submission
+3.851sCommitting number to 3E

+24m 15.814s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+143ms Scheduled

Retry attempt 7 of 10: Could not load file or assembly 'AG.A1.Scheduler.…

Enqueue at:: 1/15/2025 2:59:55 PM

+503ms Failed

Can not change the state to 'Processing': target method was not found.

System.IO.FileNotFoundException

Could not load file or assembly 'AG.A1.Scheduler.Project.Contracts, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.

System.IO.FileNotFoundException: Could not load file or assembly 'AG.A1.Scheduler.Project.Contracts, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
File name: 'AG.A1.Scheduler.Project.Contracts, Culture=neutral, PublicKeyToken=null'
   at System.Reflection.RuntimeAssembly.InternalLoad(ObjectHandleOnStack assemblyName, ObjectHandleOnStack requestingAssembly, StackCrawlMarkHandle stackMark, Boolean throwOnFileNotFound, ObjectHandleOnStack assemblyLoadContext, ObjectHandleOnStack retAssembly)
   at System.Reflection.RuntimeAssembly.InternalLoad(AssemblyName assemblyName, RuntimeAssembly requestingAssembly, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, AssemblyLoadContext assemblyLoadContext)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Hangfire.Common.TypeHelper.AssemblyResolver(String assemblyString)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Hangfire.Common.TypeHelper.CachedAssemblyResolver(AssemblyName assemblyName)
   at System.TypeNameParser.ResolveAssembly(String asmName, Func`2 assemblyResolver, Boolean throwOnError, StackCrawlMark& stackMark)
   at System.TypeNameParser.ConstructType(Func`2 assemblyResolver, Func`4 typeResolver, Boolean throwOnError, Boolean ignoreCase, StackCrawlMark& stackMark)
   at System.TypeNameParser.GetType(String typeName, Func`2 assemblyResolver, Func`4 typeResolver, Boolean throwOnError, Boolean ignoreCase, StackCrawlMark& stackMark)
   at System.Type.GetType(String typeName, Func`2 assemblyResolver, Func`4 typeResolver, Boolean throwOnError)
   at Hangfire.Common.TypeHelper.DefaultTypeResolver(String typeName)
   at Hangfire.Storage.InvocationData.DeserializeJob()

+13m 19.947s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+7ms Scheduled

Retry attempt 6 of 10: The request was canceled due to the configured Ht…

Enqueue at:: 1/15/2025 2:35:40 PM

+1m 41.360s Failed

An exception occurred during performance of the job.

System.Threading.Tasks.TaskCanceledException

The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 165
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+40ms Processing

Server:: dw0sdwk000723:3256
Worker:: a9d374bc

+176msOfflineNumberCommitTo3ECommandHandler - Started
+693msNo of records retrieved : 1
+713ms----------------------------
+723msSubmission started for Number: 621335
+919msSubmission to 3E - Started at 1/15/2025 2:20:44 PM by Sam Keegan
+1.197sCurrent item:
+1.206sProject Id: 2128221, Project Number: 313716.0003, Number Id - 621335
+1.245sTrigger Type - FN, Tax Year - 2023, State/Fed - FED
+1.254sPreparing request for submission
+1.266sCommitting number to 3E
+1m 41.316sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
+1m 41.324sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

+6m 49.453s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+7ms Scheduled

Retry attempt 5 of 10: The request was canceled due to the configured Ht…

Enqueue at:: 1/15/2025 2:20:29 PM

+1m 40.930s Failed

An exception occurred during performance of the job.

System.Threading.Tasks.TaskCanceledException

The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 165
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+23ms Processing

Server:: dw0sdwk000723:3256
Worker:: 1f68d73a

+147msOfflineNumberCommitTo3ECommandHandler - Started
+516msNo of records retrieved : 1
+522ms----------------------------
+528msSubmission started for Number: 621335
+682msSubmission to 3E - Started at 1/15/2025 2:12:13 PM by Sam Keegan
+776msCurrent item:
+784msProject Id: 2128221, Project Number: 313716.0003, Number Id - 621335
+811msTrigger Type - FN, Tax Year - 2023, State/Fed - FED
+816msPreparing request for submission
+828msCommitting number to 3E
+1m 40.873sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
+1m 40.881sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

+2m 31.127s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+46ms Scheduled

Retry attempt 4 of 10: The request was canceled due to the configured Ht…

Enqueue at:: 1/15/2025 2:12:05 PM

+1m 44.010s Failed

An exception occurred during performance of the job.

System.Threading.Tasks.TaskCanceledException

The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 165
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+24ms Processing

Server:: dw0sdwk000723:3256
Worker:: 0fb14427

+31msOfflineNumberCommitTo3ECommandHandler - Started
+1.081sNo of records retrieved : 1
+1.088s----------------------------
+1.094sSubmission started for Number: 621335
+2.190sSubmission to 3E - Started at 1/15/2025 2:07:59 PM by Sam Keegan
+2.644sCurrent item:
+2.650sProject Id: 2128221, Project Number: 313716.0003, Number Id - 621335
+2.904sTrigger Type - FN, Tax Year - 2023, State/Fed - FED
+2.912sPreparing request for submission
+2.923sCommitting number to 3E
+1m 43.697sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
+1m 43.726sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

+1m 58.190s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+313ms Scheduled

Retry attempt 3 of 10: Could not load file or assembly 'AG.A1.Scheduler.…

Enqueue at:: 1/15/2025 2:07:50 PM

+1.607s Failed

Can not change the state to 'Processing': target method was not found.

System.IO.FileNotFoundException

Could not load file or assembly 'AG.A1.Scheduler.Project.Contracts, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.

System.IO.FileNotFoundException: Could not load file or assembly 'AG.A1.Scheduler.Project.Contracts, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
File name: 'AG.A1.Scheduler.Project.Contracts, Culture=neutral, PublicKeyToken=null'
   at System.Reflection.RuntimeAssembly.InternalLoad(ObjectHandleOnStack assemblyName, ObjectHandleOnStack requestingAssembly, StackCrawlMarkHandle stackMark, Boolean throwOnFileNotFound, ObjectHandleOnStack assemblyLoadContext, ObjectHandleOnStack retAssembly)
   at System.Reflection.RuntimeAssembly.InternalLoad(AssemblyName assemblyName, RuntimeAssembly requestingAssembly, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, AssemblyLoadContext assemblyLoadContext)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Hangfire.Common.TypeHelper.AssemblyResolver(String assemblyString)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Hangfire.Common.TypeHelper.CachedAssemblyResolver(AssemblyName assemblyName)
   at System.TypeNameParser.ResolveAssembly(String asmName, Func`2 assemblyResolver, Boolean throwOnError, StackCrawlMark& stackMark)
   at System.TypeNameParser.ConstructType(Func`2 assemblyResolver, Func`4 typeResolver, Boolean throwOnError, Boolean ignoreCase, StackCrawlMark& stackMark)
   at System.TypeNameParser.GetType(String typeName, Func`2 assemblyResolver, Func`4 typeResolver, Boolean throwOnError, Boolean ignoreCase, StackCrawlMark& stackMark)
   at System.Type.GetType(String typeName, Func`2 assemblyResolver, Func`4 typeResolver, Boolean throwOnError)
   at Hangfire.Common.TypeHelper.DefaultTypeResolver(String typeName)
   at Hangfire.Storage.InvocationData.DeserializeJob()

+48.690s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+3ms Scheduled

Retry attempt 2 of 10: The request was canceled due to the configured Ht…

Enqueue at:: 1/15/2025 2:05:48 PM

+1m 41.490s Failed

An exception occurred during performance of the job.

System.Threading.Tasks.TaskCanceledException

The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 165
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+40ms Processing

Server:: dw0sdwk000723:3256
Worker:: 1f68d73a

+70msOfflineNumberCommitTo3ECommandHandler - Started
+1.100sNo of records retrieved : 1
+1.107s----------------------------
+1.113sSubmission started for Number: 621335
+1.183sSubmission to 3E - Started at 1/15/2025 2:03:28 PM by Sam Keegan
+1.293sCurrent item:
+1.299sProject Id: 2128221, Project Number: 313716.0003, Number Id - 621335
+1.338sTrigger Type - FN, Tax Year - 2023, State/Fed - FED
+1.348sPreparing request for submission
+1.354sCommitting number to 3E
+1m 41.424sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
+1m 41.441sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

+33.413s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+7ms Scheduled

Retry attempt 1 of 10: The request was canceled due to the configured Ht…

Enqueue at:: 1/15/2025 2:03:15 PM

+1m 42.250s Failed

An exception occurred during performance of the job.

System.Threading.Tasks.TaskCanceledException

The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 165
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+20ms Processing

Server:: dw0sdwk000723:3256
Worker:: a9d374bc

+25msOfflineNumberCommitTo3ECommandHandler - Started
+1.221sNo of records retrieved : 1
+1.230s----------------------------
+1.237sSubmission started for Number: 621335
+1.987sSubmission to 3E - Started at 1/15/2025 2:01:13 PM by Sam Keegan
+2.113sCurrent item:
+2.121sProject Id: 2128221, Project Number: 313716.0003, Number Id - 621335
+2.141sTrigger Type - FN, Tax Year - 2023, State/Fed - FED
+2.148sPreparing request for submission
+2.154sCommitting number to 3E
+31.220sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
+31.231sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
+1m 42.193sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
+1m 42.203sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

+13ms Enqueued

Triggered by recurring job scheduler

Queue:: a1-numbers

CurrentCulture	`"en-US"`
CurrentUICulture	`"en-US"`
RecurringJobId	`"NUMBER-Offline-Commit-Number-To-3E-Task"`
RetryCount	`10`
Time	`1736949671`

Parameters

Requeue Delete State

1/15/2025 6:50:04 PM (+32.613s) Failed

System.Net.Http.HttpRequestException

+30ms Processing

+1h 50m Enqueued

+3ms Scheduled

+1m 41.867s Failed

System.Threading.Tasks.TaskCanceledException

+20ms Processing

+1h 10m Enqueued

+16ms Scheduled

+1m 42.957s Failed

System.Threading.Tasks.TaskCanceledException

+77ms Processing

+42m 49.276s Enqueued

+10ms Scheduled

+1m 43.970s Failed

System.Threading.Tasks.TaskCanceledException

+40ms Processing

+24m 15.814s Enqueued

+143ms Scheduled

+503ms Failed

System.IO.FileNotFoundException

+13m 19.947s Enqueued

+7ms Scheduled

+1m 41.360s Failed

System.Threading.Tasks.TaskCanceledException

+40ms Processing

+6m 49.453s Enqueued

+7ms Scheduled

+1m 40.930s Failed

System.Threading.Tasks.TaskCanceledException

+23ms Processing

+2m 31.127s Enqueued

+46ms Scheduled

+1m 44.010s Failed

System.Threading.Tasks.TaskCanceledException

+24ms Processing

+1m 58.190s Enqueued

+313ms Scheduled

+1.607s Failed

System.IO.FileNotFoundException

+48.690s Enqueued

+3ms Scheduled

+1m 41.490s Failed

System.Threading.Tasks.TaskCanceledException

+40ms Processing

+33.413s Enqueued

+7ms Scheduled

+1m 42.250s Failed

System.Threading.Tasks.TaskCanceledException

+20ms Processing

+13ms Enqueued

1/15/2025 2:01:11 PM Created

State