0

We have an existing reporting application which is migrating to Google Bigquery and for detail page we have to implement the pagination concept.

Currently we are creating a temp table to store the queried data and query the temp-table for pagination. But scalability wise as well as cost wise it is not feasible for us, as our user base is high and we have to build these temp tables for each user (as data for each user differ with different roles) and delete it every interval.

Is there a way we can use the google created temp table which store the data for 24 hrs and use it for pagination. Or is there any other better approach which can be implemented to deal with this situation.

Request someone to suggest a better approach which can be implemented for concurrent users in bigquery.

We have done this POC to implement the BQ Pagination.

TableId tableId = TableId.of("gcp-projectID","AppS_DV3", "cm1_temp");
Table table = bigquery.getTable(tableId);
if(table  != null && table.exists()) {
    QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(queryBuilder.toString())
    //.setDestinationTable(tableId)
    .build();
    bigquery.query(queryConfig);
}else {
    QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(queryBuilder.toString())
    .setDestinationTable(tableId)
    .build();
    bigquery.query(queryConfig);
}
int pageNo = Integer.parseInt(requestObject.get("pageNo").toString());
TableResult results =
    bigquery.listTableData( tableId,BigQuery.TableDataListOption.startIndex(pageNo),BigQuery.TableDataListOption.pageSize(5));
JSONArray jarray = new JSONArray();
for (FieldValueList row : results.getValues()) {
    for (FieldValue fv : row) {
        System.out.println(fv.getValue());
        JSONObject obj = (JSONObject) new JSONParser().parse(fv.getValue().toString());
        jarray.add(obj);
    }
}
rapati
  • 1

0 Answers0