0

I'm newbie in async. And making WPF app for scraping and API calls purposes. WPF's UI is needed only for service monitoring and settings control, so all services will run simultaneously in the background. And most of them is doing similar work. For this one I need to implement strategy like this:

  1. Start worker on threadpool
  2. Worker must send request and process response from Website
    2.1 If response processed and some new data appeared - raise an event
    2.2 If request failed - handle an error
    2.3 If there are many error percent for last x requests - stop worker
  3. No matter the last request completed/failed/running we must send another request
  4. Another request should be sent not earlier than the set delay but should not exceed the delay too much (as far as possible).
private _workTask;
private List<ScrapeParameters> _scrapeParams = new();
public event EventHandler<ScrapedEventArgs>? NewDataScraped;

//Can I run Worker like this?
public void RunWorker()
{
    if(_workTask.IsCompleted)
        _workTask = WorkAsync(_token)
}

private async Task WorkAsync(CancellationToken cancelToken)
{
    List<Task> processTasks = new();
    while(true)
    {
        if(cancelToken.IsCancellationRequested) return;
        
        //Delay could be from 0.5 second to any value
        var delayTask = Task.Delay(WorkerDelay);
        
        var completedTasks = processTasks.Where(t => t.IsCompleted)
        var setToHandle = new HashSet<Task>(completedTasks);

        foreach(var task in setToHandle)
        {
            //Theoretical logic to handle errors and completion
            if(task.IsFaulted)
                HandleFaultedTask(task);
            else
                CountCompleted();

            processTasks.Remove(task);
        }
        
        //Theoretical logic to obtain the desired parameters.
        var currParameters = GetParameters();
        processTasks.Add(ProcessAsync(currParameters, cancelToken));
        await delayTask;
    }
    
}
//This method usually takes around 2-4 seconds
private async Task ProcessAsync(ScrapeParameters parameters CancellationToken cancelToken)
{
    //Some work with http requests 
    var response = await Client.GetAsync(parameters.ToUri());
    ...
    //Processing response
    ...
    if(newData != null)
        NewDataScraped?.Invoke(new(newData));
}

Does my implementation matches the TAP pattern?
Especially I would like to focus on RunWorker() and setToHandle

achies
  • 43
  • 1
  • 5
  • Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. – Community Jul 26 '22 at 12:54
  • See if [Best way to check a url is valid every second (or less), with Task await](https://stackoverflow.com/a/72900377/5438626) is helpful You would process your response in the `onPingSuccess` method shown. – IVSoftware Jul 26 '22 at 12:58
  • @IVSoftware thanks. I realized that its better to ping website and if its down stop services working with it. But I also have to check errors inside faulted tasks because sometimes website send 200 code and provides wrong information/error page (site internal problems). – achies Jul 26 '22 at 13:17
  • Thx I think I understand and in the comments of that answer there's some discussion of using this [answer](https://stackoverflow.com/a/7524060/5438626) and substituting the http request instead of a simple ping. So there still might be something helpful there...or not :) – IVSoftware Jul 26 '22 at 13:21
  • *"No matter the last request completed/failed/running we must send another request. Another request should be sent not earlier than the set delay."* -- From this I understand that new requests should be sent every X seconds, and also every time a request completes. Which means that the number of active requests will increase linearly and steadily over time. Is my understanding correct? Also is this the desirable behavior? – Theodor Zoulias Jul 26 '22 at 15:32
  • *"No matter the last request completed/failed/running we must send another request"* -- Is the next request related in any way with the completed request? – Theodor Zoulias Jul 26 '22 at 15:35
  • *"If response processed and some new data appeared - raise an event."* -- Is there any requirement regarding the invocation of this event? For example should it be raised on a specific thread? If not, should the invocation be synchronized, or you are OK with the event being raised on multiple threads concurrently and in parallel with other invocations, in an overlapping manner? – Theodor Zoulias Jul 26 '22 at 15:43
  • @TheodorZoulias 1. No, next request is fully independent from previous. 2. After this event is being raised another service in async manner will read `EventArgs` and start 2 another `Task`s (2 for current version). First one makes some API calls to website. The second notifies me at messanger. So overlapping is not a problem. Regarding the synchronization of the invocation, you mean that the next invocation does not start before the end of the previous one, am I right? – achies Jul 26 '22 at 16:03

0 Answers0