Displaying more than 10000 rows using Core Reporting Google API v4 ( Java)

Question

I'm fetching Google Analytics data using Core Reporting API v4. I'm able to capture at most 10,000 records for a given combination of Dimensions & Metrics. My question is that if my query can produce more than 10,000 search results then how can I fetch all those records? I have gone through the documentation and found that in a single request we can't access more than 10,000 records by setting the properties of ReportRequest object.

ReportRequest request = new ReportRequest()
    .setDateRanges(Arrays.asList(dateRange)) 
    .setViewId(VIEW_ID)
    .setDimensions(Arrays.asList(dimension))
    .setMetrics(Arrays.asList(metric))
    .setPageSize(10000); 

How can we enable multiple requests in a single run depending upon the number of search-results that can be obtained.

For example : If my query can return 35,000 records then there should be 4 requests (10,000,10,000, 10,000 & 3,500) managed internally.

Please look into this and facilitate me some guidance. Thanks in Advance.

Answer 1

The Analytics Core Reporting API returns a maximum of 10,000 rows per request, no matter how many you ask for.

If the request you are making will generate more then 10000 rows then there will be additional rows you can request. The response returned from the first request will contain a parameter called nextPageToken which you can use to request the next set of data.

You will have to dig around the Java library the only documentation on how to do it I have found is HTTP.

POST https://analyticsreporting.googleapis.com/v4/reports:batchGet
{
  "reportRequests":[
  {
    ...
    # Taken from `nextPageToken` of a previous response.
    "pageToken": "XDkjaf98234xklj234",
    "pageSize": "10000",
  }]
}

Answer 2

Here's a stable and extensively tested solution in Java. It is a recursive solution that stores every 10000 results batch (if any) and recalls itself until finds a null nextToken. In this specific solution every 10000 results batch is saved into a csv and then a recursive call is performed! Note that the first time this function called from somewhere outside, the nextPageToken is also null!! Focus on the recursive rationale and the null value check!

private static int getComplexReport(AnalyticsReporting service,int 
reportIndex,java.lang.String startDate,String endDate,ArrayList<String>
metricNames,ArrayList<String> dimensionNames,String pageToken)    
throws IOException

ReportRequest req = createComplexRequest(startDate,endDate,metricNames,dimensionNames,pageToken);

ArrayList<ReportRequest> requests = new ArrayList<>();
requests.add(req);

// Create the GetReportsRequest object.
GetReportsRequest getReport = new GetReportsRequest()
    .setReportRequests(requests);

// Call the batchGet method.
GetReportsResponse response = service.reports().batchGet(getReport).execute();
      //printResponse(response);


saveBatchToCsvFile("dummy_"+startDate+"_"+endDate+"_"+Integer.toString(reportIndex)+".csv",startDate+"_"+endDate,response,metricNames,dimensionNames);
String nextToken = response.getReports().get(0).getNextPageToken();
//System.out.println(nextToken);
if(nextToken!=null)
    return getComplexReport(service,reportIndex+1,"2016-06-21","2016-06-21",metricNames,dimensionNames,nextToken);

return reportIndex; 
} 

Answer 3

var reportRequest = new ReportRequest
{
    DateRanges = new List<DateRange> { dateRange },
    Dimensions = new List<Dimension> { date, UserId, DeviceCategory},
    Metrics = new List<Metric> { sessions },
    ViewId = view,
    PageSize = 400000
};