1

I am working on implementing lazy loading utilizing bwu-grid's dataview implementation in Dart. My server is implemented with Google App Engine (Java) over cloudendpoint API. When returning over 1,000+ rows, if no fetch option is set, I get an exception thrown.

I currently have JQuery's pagination in the bottom of dataview. I want to implement lazy loading utilizing jdo's query cursor in such way that upon changing the page, it loads the particular segment of the dataset.

I looked over dataview class implementation and found the following listener I can potentially utilize:

  async.Stream<core.PagingInfoChanged> get onBwuPagingInfoChanged =>
      _eventBus.onEvent(core.Events.PAGING_INFO_CHANGED);

The implementation plan is the following:

  1. Query for 1st page of data and total row count, update the datagrid with total row count
  2. Listen for the page changing event
  3. Make cloud endpoint using query cursor that returns the data segment
  4. Insert the data segment into the grid and refresh the data-grid

How is my implementation plan? What else do I have to consider when implementing this way? Is there any examples showing lazy loading support in bwu-datagrid? Any suggestions would be appreciated.

    W 12:06:21.550 com.google.appengine.api.datastore.BaseQueryResultsSource logChunkSizeWarning: This query does not have a chunk size set in FetchOptions and has returned over 1000 results.  If result sets of this size are common for this query, consider setting a chunk size to improve performance.
        To disable this warning set the following system property in appengine-web.xml (the value of the property doesn't matter): 'appengine.datastore.disableChunkSizeWarning'
    E 12:07:19.760 com.google.api.server.spi.SystemService invokeServiceMethod: exception occurred while calling backed method
      javax.jdo.JDODataStoreException: Datastore Failure
        at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:421)
        at org.datanucleus.api.jdo.JDOPersistenceManager.internalClose(JDOPersistenceManager.java:367)
        at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.releasePersistenceManager(JDOPersistenceManagerFactory.java:1106)
        at org.datanucleus.api.jdo.JDOPersistenceManager.close(JDOPersistenceManager.java:343)
        at com.sports_tla.MLBMatchEndpoint.listMLBMatch(MLBMatchEndpoint.java:81)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:45)
        at com.google.api.server.spi.SystemService.invokeServiceMethod(SystemService.java:359)
        at com.google.api.server.spi.SystemServiceServlet.execute(SystemServiceServlet.java:160)
        at com.google.api.server.spi.SystemServiceServlet.doPost(SystemServiceServlet.java:118)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
        at com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:437)
        at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:444)
        at com.google.tracing.CurrentContext.runInContext(CurrentContext.java:220)
        at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:308)
        at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:300)
        at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:441)
        at java.lang.Thread.run(Thread.java:724)
      NestedThrowablesStackTrace:
      com.google.appengine.api.datastore.DatastoreFailureException: Unexpected failure
        at com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.java:78)
        at com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java:62)
        at com.google.appengine.api.datastore.DatastoreServiceImpl.get(DatastoreServiceImpl.java:41)
        at com.google.appengine.datanucleus.WrappedDatastoreService.get(WrappedDatastoreService.java:60)
        at com.google.appengine.datanucleus.FetchFieldManager.lookupOneToOneChild(FetchFieldManager.java:581)
        at com.google.appengine.datanucleus.FetchFieldManager.fetchRelationField(FetchFieldManager.java:541)
        at com.google.appengine.datanucleus.FetchFieldManager.fetchObjectField(FetchFieldManager.java:405)
        at org.datanucleus.state.AbstractStateManager.replacingObjectField(AbstractStateManager.java:2353)
        at com.sports_tla.MLBMatch.jdoReplaceField(MLBMatch.java)
        at com.sports_tla.MLBMatch.jdoReplaceFields(MLBMatch.java)
        at org.datanucleus.state.JDOStateManager.replaceFields(JDOStateManager.java:1935)
        at org.datanucleus.state.JDOStateManager.replaceFields(JDOStateManager.java:1962)
        at com.google.appengine.datanucleus.EntityUtils$1.fetchFields(EntityUtils.java:974)
        at org.datanucleus.state.JDOStateManager.loadFieldValues(JDOStateManager.java:764)
        at org.datanucleus.state.JDOStateManager.initialiseForHollow(JDOStateManager.java:205)
        at org.datanucleus.state.StateManagerFactory.newForHollowPopulated(StateManagerFactory.java:89)
        at org.datanucleus.state.ObjectProviderFactory.newForHollowPopulated(ObjectProviderFactory.java:75)
        at org.datanucleus.ObjectManagerImpl.findObject(ObjectManagerImpl.java:2882)
        at com.google.appengine.datanucleus.EntityUtils.entityToPojo(EntityUtils.java:1014)
        at com.google.appengine.datanucleus.query.DatastoreQuery$2.apply(DatastoreQuery.java:229)
        at com.google.appengine.datanucleus.query.DatastoreQuery$2.apply(DatastoreQuery.java:226)
        at com.google.appengine.datanucleus.query.LazyResult.resolveNext(LazyResult.java:96)
        at com.google.appengine.datanucleus.query.LazyResult.resolveAll(LazyResult.java:121)
        at com.google.appengine.datanucleus.query.StreamingQueryResult.closingConnection(StreamingQueryResult.java:90)
        at org.datanucleus.store.query.AbstractQueryResult.disconnect(AbstractQueryResult.java:108)
        at com.google.appengine.datanucleus.query.StreamingQueryResult.disconnect(StreamingQueryResult.java:78)
        at com.google.appengine.datanucleus.query.JDOQLQuery$1.managedConnectionPreClose(JDOQLQuery.java:199)
        at com.google.appengine.datanucleus.DatastoreConnectionFactoryImpl$DatastoreManagedConnection.close(DatastoreConnectionFactoryImpl.java:147)
        at org.datanucleus.store.connection.ConnectionManagerImpl.closeAllConnections(ConnectionManagerImpl.java:181)
        at org.datanucleus.store.AbstractStoreManager$1.preClose(AbstractStoreManager.java:260)
        at org.datanucleus.ObjectManagerImpl.close(ObjectManagerImpl.java:1112)
        at org.datanucleus.api.jdo.JDOPersistenceManager.internalClose(JDOPersistenceManager.java:359)
        at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.releasePersistenceManager(JDOPersistenceManagerFactory.java:1106)
        at org.datanucleus.api.jdo.JDOPersistenceManager.close(JDOPersistenceManager.java:343)
        at com.sports_tla.MLBMatchEndpoint.listMLBMatch(MLBMatchEndpoint.java:81)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:45)
        at com.google.api.server.spi.SystemService.invokeServiceMethod(SystemService.java:359)
        at com.google.api.server.spi.SystemServiceServlet.execute(SystemServiceServlet.java:160)
        at com.google.api.server.spi.SystemServiceServlet.doPost(SystemServiceServlet.java:118)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
      <continued in next message>
LuxuryMaster
  • 577
  • 1
  • 4
  • 11
  • I don't know what you mean with "fetch option". Would be interesting what the exception is. BWU Datagrid is a port of https://github.com/mleibman/SlickGrid. SlickGrid contains an example for such an use case. Sadly it's one of the examples I haven't ported to Dart yet and I haven't used the grid this way yet. (see also https://github.com/mleibman/SlickGrid/blob/master/examples/example6-ajax-loading.html) – Günter Zöchbauer Mar 11 '15 at 07:12
  • Hey Gunter, thanks for the response. I attached the log for the exception for your view. Fetch options in chunk size is what I'm referring to btw. – LuxuryMaster Mar 11 '15 at 18:38
  • This looks like a server side (appengine/JDO) problem. I think I can't help you with that, only about Dart and BWU Datagrid. – Günter Zöchbauer Mar 11 '15 at 18:42
  • Maybe the way I asked could have been misleading. I wasn't asking help for JDO issues. I only included exception because you were curious about it - I know how to solve that. I was simply asking for potential BWU datagrid usage and asking how my implementation plan looks – LuxuryMaster Mar 11 '15 at 22:40
  • I guess your strategy is right. I hope to find time soon to port the AJAX example to learn what ideas the original creator had for this scenario. – Günter Zöchbauer Mar 12 '15 at 05:00
  • Sounds good. For the time being, I successfully implemented lazy loading asynchronously in the background although this might not be the best solution. Let me know if this can be improved later. – LuxuryMaster Mar 14 '15 at 09:36
  • sure, as soon as I get there – Günter Zöchbauer Mar 14 '15 at 09:37

0 Answers0