0

I am working on a scenario where I'm calling a Web API and doing a computation based on the results in the data section of the JSON. However, the response of API is paginated, like so:

{
  "page": 1,
  "per_page": 5,
  "total": 500,
  "total_pages": 100,
  "data": [
    {
      "competition": "UEFA Champions League",
      "year": 2011,
      "round": "GroupH",
      "team1": "Barcelona",
      "team2": "AC Milan",
      "team1goals": "2",
      "team2goals": "2"
    },
    {
      "competition": "UEFA Champions League",
      "year": 2011,
      "round": "GroupG",
      "team1": "APOEL Nikosia",
      "team2": "Zenit St. Petersburg",
      "team1goals": "2",
      "team2goals": "1"
    },
    {
      "competition": "UEFA Champions League",
      "year": 2011,
      "round": "GroupF",
      "team1": "Borussia Dortmund",
      "team2": "Arsenal",
      "team1goals": "1",
      "team2goals": "1"
    }
}

For me to get to the next page to get more data, I have to pass the page number in the QueryString like so:

https://jsonmock.hackerrank.com/api/football_matches?year=2011&page=2

So I have put this in a for loop to iterate thru those 100 pages to compute the result. And as I am iterating thru the 100 pages, I get an execution Time Limit Exceeded (TLE) error before the code could get to the 7th page in the loop. Here is my code:

public class Program
{
    public static void Main()
    {       
        int year = 2011;
        var result = Result.getNumDraws(year);
        Console.WriteLine("Match drawn = " + result);
    }
}

class Result
{   
        public static int getNumDraws(int year)
    {
        var totalDrawn = 0;
        HttpClient client = new HttpClient();
        var baseURL = "https://jsonmock.hackerrank.com/api/football_matches?year=" + year.ToString();
        var current = client.GetAsync(baseURL).Result;
        var result = JsonConvert.DeserializeObject<ApiResponse>(current.Content.ReadAsStringAsync().Result);
    
        for (int i = 1; i <= result.TotalPages; i++)
        {
            Console.WriteLine("iterating thru pages = " + i);
            var response =  GetMatchInfo(year, i);

            foreach (var match in response.data)
            {
                if (match.team1goals == match.team2goals)
                    totalDrawn++;
            }
        }

        return totalDrawn;
    }

    public static ApiData GetMatchInfo(int year, int page)
    {
        var url = "https://jsonmock.hackerrank.com/api/football_matches?year=" + year.ToString() + "&page=" + page;
        using (HttpClient client = new HttpClient())
        {
            
            var currentData = client.GetAsync(url).Result;          
            var response = JsonConvert.DeserializeObject<ApiData>(currentData.Content.ReadAsStringAsync().Result);
            return response;        
        }
    }
}

public class ApiResponse
{
    public string Page { get; set; }

    public int Total { get; set; }

    [JsonProperty("total_pages")]
    public int TotalPages { get; set; }
}

public class ApiData
{
    public List<Goal> data { get; set; }
}

public class Goal
{
    public int team1goals { get; set; }

    public int team2goals { get; set; }
}

Could you please guide me on what tweak or update I need to make in my code to address the Execution Time Limit error?

Thank you.

theITvideos
  • 1,462
  • 2
  • 18
  • 29
  • 1
    How are you getting the time limit exceeded error - as a return value to GetAsync or an exception? May be the API endpoint is throttling too many calls within short period of time. – Anand Sowmithiran Jan 10 '22 at 11:08
  • and why are you not using *await* GetAsync()... ? see this [post](https://stackoverflow.com/questions/10343632/httpclient-getasync-never-returns-when-using-await-async) – Anand Sowmithiran Jan 10 '22 at 11:11
  • And you should not instantiate this many `HttpClient` objects, you should declare one as static at application level and reuse it. – Anand Sowmithiran Jan 10 '22 at 11:19
  • I believe the API itself is not throttling, I am submitting this code in an online portal that has a maximum execution time of about 5 seconds. The code executes fine for the first couple of seconds and errors out saying the execution time limit was exceeded. – theITvideos Jan 10 '22 at 11:32
  • You need to add await , and have only 1 instance of HttpClient as I mentioned above. Even otherwise, calling that HTTP GET 100 times is going to take time and will not finish within 5 seconds. Please get to know more about what that online portal's restrictions are, looks like contest , so it is not going to be simple answer as you implemented. – Anand Sowmithiran Jan 10 '22 at 11:41
  • You may also check whether you can instruct the API to return more items within one page (aka _page size_) – mu88 Jan 10 '22 at 12:20
  • thank mu88 for the comment, this is an API from HackerRank app and can't really update API to return all or more items in a single call. – theITvideos Jan 10 '22 at 12:34

1 Answers1

0

OK I have now updated the code to include async and await and have defined a single static HttpClient object as shown here:

public class Program {

    public static void Main () {        
        int year = 2011;
        var result = Result.getNumDraws (year);
        Console.WriteLine ("Match drawn = " + result);
    }
}

class Result {
    static HttpClient client = new HttpClient ();

    public static async Task<int> getNumDraws (int year) {
        var totalDrawn = 0;
        
        var baseURL = "https://jsonmock.hackerrank.com/api/football_matches?year=" + year.ToString ();

        var current = await client.GetAsync (baseURL);
        var result = JsonConvert.DeserializeObject<ApiResponse> (current.Content.ReadAsStringAsync ().Result);

        for (int i = 1; i <= result.TotalPages; i++) {
            Console.WriteLine ("iterating thru pages = " + i);
            var response = GetMatchInfo (year, i);
            foreach (var match in response.Result.data) {
                if (match.team1goals == match.team2goals)
                    totalDrawn++;
            }
        }

        return totalDrawn;
    }

    public static async Task<ApiData> GetMatchInfo (int year, int page) {
        var url = "https://jsonmock.hackerrank.com/api/football_matches?year=" + year.ToString () + "&page=" + page;

        var currentData = await client.GetAsync (url);

        var response = JsonConvert.DeserializeObject<ApiData> (currentData.Content.ReadAsStringAsync ().Result);
        return response;

    }
}

public class ApiResponse {
    public string Page { get; set; }

    public int Total { get; set; }

    [JsonProperty ("total_pages")]
    public int TotalPages { get; set; }
}

public class ApiData {
    public List<Goal> data { get; set; }
}

public class Goal {
    public int team1goals { get; set; }

    public int team2goals { get; set; }
}

When I execute the code in my console app and set a breakpoint, I see this information in the result variable: enter image description here

I am expecting the result variable to hold the total computed value but it shows some Task-related information because the calls are still being made to fetch the data from the Web API? What do we do here to get the total computed value in the result variable?

theITvideos
  • 1,462
  • 2
  • 18
  • 29
  • hi Anand, I have posted the updated code here, please let me know if there's anything I need to update here to get the Web API computed details in the `result` variable of the Main method. – theITvideos Jan 10 '22 at 14:59