Facing Critical Performance issue in Primefaces 4 & 5

Question

I am working on a project which deal with heavy data sets. I am using Primefaces 4 & 5, spring and hibernate. I have to to display a very huge datasets such as min 3000 rows with 100 columns with various features such as sorting, filtering, row-expansion etc. My problem is, my applications took 8 to 10 mins to show the whole page as well as other functionalities(sorting, filtering ) also takes a lot time. My client is not happy at all. However I can use pagination for this but again My client do not want paging. So I decided to use livescroll but unfortunately I failed to implement livescroll with lazyload or without lazyload as there were bugs in PF regarding livescroll. also i have posted this question here earlier but no solution found.

This performance issue is very critical and show stopper for me. To show 3000 rows with 100 columns, the size of the page which is getting loaded is ~10MB. I have calculated the time consumed by various life-cycles of of JSF, using Phase-listener I figure out that its Browser who is taking time to parse the response rendered by jsf. To complete the all phases my application took only 25 sec. At minimal I want to increase the performance of my project. Please share any idea, suggestion and anything which could help to overcome this problem

Note: There is no database manipulations in getters and setters as well as no complex business logic.

UPDATE : This is my datatable without lazyload:

<p:dataTable 
                style="width:100%"
                id="cdTable"
                selection="#{controller.selectedArray}"
                resizableColumns="true" 
                draggableColumns="true" 
                var="cd"
                value="#{controller.cdDataModel}"
                editable="true" 
                editMode="cell"
                selectionMode="multiple"
                rowSelectMode="add"
                scrollable="true"
                scrollHeight="650"
                rowKey="#{cd.id}"
                rowIndexVar="rowIndex"
                styleClass="screenScrollStyle"
                liveScroll="true"
                scrollRows="50"
                filterEvent="enter"
                widgetVar="dt4"
                >

Here everything is working except filtering. Once I filter then first page is displayed but unable to sort or livescroll on datatable. Note this I have tested in Primefaces5.

2nd Approch

With lazyload with same datatable 1) When I add rows="100" livescroll happens but problem with row-editing, row-expansion but filter & sorting works. 2) When I remove rows livescroll works with row-editing, row-expansion etc but filter & sorting dont work.

My LazyLoadModel is as follows

public class MyDataModel extends LazyDataModel<YData> 
         {

    @Override
    public List<YData> load(int first, int pageSize,
            List<SortMeta> multiSortMeta, Map<String, Object> filters) {
        System.out.println("multisort wala load");
        return super.load(first, pageSize, multiSortMeta, filters);
    }



    /**
     * 
     */
    private static final long serialVersionUID = 1L;
    private List<YData> datasource;

    public YieldRecBondDataModel() {
    }

    public YieldRecBondDataModel(List<YData> datasource) {
         this.datasource = datasource;
    }




    @Override
    public YData getRowData(String rowKey) {
        // In a real app, a more efficient way like a query by rowKey should be
        // implemented to deal with huge data

    //  List<YData> yList = (List<YData>) getWrappedData();

        for (YData y : datasource) 
        {
            System.out.println("datasource :"+datasource.size());
            if(y.getId()!=null)
            {
            if (y.getId()==(new Long(rowKey)))
            {
                return y;
            }
            }
        }

        return null;
    }

    @Override
    public Object getRowKey(YData y) {
        return y.getId();
    }






      @Override
        public void setRowIndex(int rowIndex) {
            /*
             * The following is in ancestor (LazyDataModel):
             * this.rowIndex = rowIndex == -1 ? rowIndex : (rowIndex % pageSize);
             */
            if (rowIndex == -1 || getPageSize() == 0) {
                super.setRowIndex(-1);
            }
            else
                super.setRowIndex(rowIndex % getPageSize());
        }



      @Override
        public List<YData> load(int first, int pageSize, String sortField, SortOrder sortOrder, Map<String,Object> filters) {
            List<YData> data = new ArrayList<YData>();
            System.out.println("sort order : "+sortOrder);
            //filter
            for(YData yInfo : datasource) {
                boolean match = true;

                for(Iterator<String> it = filters.keySet().iterator(); it.hasNext();) {
                    try {
                        String filterProperty = it.next();
                        String filterValue = String.valueOf(filters.get(filterProperty));
                        Field yField = yInfo.getClass().getDeclaredField(filterProperty);
                        yField.setAccessible(true);
                        String fieldValue = String.valueOf(yField.get(yInfo));
                        if(filterValue == null || fieldValue.startsWith(filterValue)) {
                            match = true;
                        }
                        else {
                            match = false;
                            break;
                        }
                    } catch(Exception e) {
                        e.printStackTrace();
                        match = false;
                    }
                }

                if(match) {
                    data.add(yInfo);
                }
            }

            //sort
            if(sortField != null) {
                Collections.sort(data, new LazySorter(sortField, sortOrder));
            }

            int dataSize = data.size();
            this.setRowCount(dataSize);

            //paginate
            if(dataSize > pageSize) {
                try {

                    List<YData> subList = data.subList(first, first + pageSize);
                    return subList;
                }
                catch(IndexOutOfBoundsException e) {
                    return data.subList(first, first + (dataSize % pageSize));
                }
            }
            else
                return data;
        }

    @Override
    public int getRowCount() {
        // TODO Auto-generated method stub
        return super.getRowCount();
    }

    }

I am fade up with these issues and becomes show stopper for me. Even i tried Primefaces 5

Answer 1

If your data is loaded from db i suggest you to do a better LazyDataModel like:

 public class ElementiLazyDataModel extends LazyDataModel<T> implements Serializable {

        private Service<T> abstractFacade;


        public ElementiLazyDataModel(Service<T> abstractFacade) {
            this.abstractFacade = abstractFacade;

        }

        public Service<T> getAbstractFacade() {
            return abstractFacade;
        }

        public void setAbstractFacade(Service<T> abstractFacade) {
            this.abstractFacade = abstractFacade;
        }

        @Override
        public List<T> load(int first, int pageSize, String sortField, SortOrder sortOrder, Map<String, Object> filters) {
            PaginatedResult<T> pr = abstractFacade.findRange(new int[]{first, first + pageSize}, sortField, sortOrder, filters);

            setRowCount(new Long(pr.getTotalItems()).intValue());

            return pr.getItems();
        }

    }

The service is some kind of backend communication (like an EJB) injected in the ManagedBean that use this model.

The service for pagination may be like this:

@Override
    public PaginatedResult<T> findRange(int[] range, String sortField, SortOrder sortOrder, Map<String, Object> filters) {

        final Query query = getEntityManager().createQuery("select x from " + entityClass.getSimpleName() + " x")
                .setFirstResult(range[0]).setMaxResults(range[1] - range[0] + 1);

        // Add filter sort etc.

        final Query queryCount = getEntityManager().createQuery("select count(x) from " + entityClass.getSimpleName() + " x");
        // Add filter sort etc.

        Long rowCount = (Long) queryCount.getSingleResult();

        List<T> resultList = query.getResultList();

        return new PaginatedResult<T>(resultList, rowCount);
    }

Note that you have to do the paginated query (with jpa like this the orm do the query for you, but if you don't use orm have to do paginated query, for oracle look at TOP-N query, for example: http://oracle-base.com/articles/misc/top-n-queries.php)

Remember your return obj must be contains also the total record as a fast count:

public class PaginatedResult<T> implements Serializable {

    private List<T> items;
    private long totalItems;

    public PaginatedResult() {
    }

    public PaginatedResult(List<T> items, long totalItems) {
        this.items = items;
        this.totalItems = totalItems;
    }

    public List<T> getItems() {
        return items;
    }

    public void setItems(List<T> items) {
        this.items = items;
    }

    public long getTotalItems() {
        return totalItems;
    }

    public void setTotalItems(long totalItems) {
        this.totalItems = totalItems;
    }
}

All this is useful if your database table is correctly setup, pay aptention to the execution plan of the possible query and add the right index.

Hope to give some hint to improve you performance

In the end, remember to your final user that the human eyes can't see more that 10-20 record at once, so it is very useless to have thousand record in a page.

Answer 2

You have used the default load implementation which is used in the showcases of Primefaces. This is not the correct implementation for your case where you load your data from a database.

The load method should use the correct query with consideration of :

1) the filter fields that are used, example:

String query = "select e from Entity e where lower(e.f1) like lower('" + filters.get(key) + "'%) and..., etc. for the other fields

2) the sorting columns that are used, example:

query.append("order by ").append(sortField).append(" ").append(SortOrder.ASCENDING.name() ? "" : sortOrder.substring(0, 4)),..., etc. for the other columns.

3) The total count of your query WITH 1) attached to it. Example:

Long totalCount = (Long) entityManager.createQuery("select count(*) from Entity e where lower(e.f1) like lower('filterKey1%') and lower(e.f2) like lower('filterKey2%') ...").getSingleResult();