Hibernate saveOrUpdate large data

Question

I am trying to insert or update a large data with Hibernate. I have a list contains 350k objects and when I use Hibernate saveOrUpdate(), it takes hours to insert all the data.

I am using the code below for this operation. My development environment is JDK1.4 and Oracle Database.

public void saveAll(List list)throws HibernateException{
    Session session = HibernateUtil.getEYSSession();
    Iterator it = list.iterator();
    int i = 0;
    while(it.hasNext()){ 
        i++;
        Object obj = it.next();
        session.saveOrUpdate(obj);
        if (i % 50 == 0) { session.flush(); session.clear(); }
    }
}

I am using batch update and also set hibernate.jdbc.batch_size property 50 but it didn't help.

My object has one-to-one relation with another object so in this case using StatelessSession might be a problem. Because StatelessSession does not cascade to composed objects I think.

Do you have any ideas about how to increase the performance of saveOrUpdate() operation in this case?

Thanks.

Which database platform are you using and are you auto-generating identifiers for your entries? — Alex Barnes, Jun 13 '13 at 10:09

score 0 · Accepted Answer · answered Jun 13 '13 at 09:17

For every saveOrUpdate operation Hibernate generates a select statement as well. In my opinion updating or inserting huge data using Hibernate's saveOrUpdate is not at all a good option. I would suggest using either a stored procedure or move to SpringJDBCTemplate.

score 0 · Answer 2 · answered Jun 13 '13 at 09:21

I found that separating the instances where the save() method or the Hibernate merge() had to be used provided slightly better performance. But, I remember saving and updating 100,000 objects and it also took quite a long time. For that reason a stored procedure may be the more efficient mechanism.

score 0 · Answer 3 · answered Jun 13 '13 at 09:21

0

You can use futureTask to complete your database inserting. and continue your program.

This will not reduce the time but program doesn't wait for completing insert process.

answered Jun 13 '13 at 09:21

Ruchira Gayan Ranaweera

34,993
17
75
115

This answer is somewhat out of context. The question is about improving the performance of db related code in case of a large number of records. Your answer will definitely improve the app performance but doesn't cater as the solution. – Himanshu Bhardwaj Jun 13 '13 at 09:40
Session session = sessionFactory.openStatelessSession(); Transaction tx = session.beginTransaction(); int i = 0; while(it.hasNext()){ i++; Object obj = it.next(); session.save(obj) if (i % 50 == 0) { session.flush(); session.clear();} – Ruchira Gayan Ranaweera Jun 13 '13 at 09:49
I am using JDK 1.4 and I used threads for saving but it also didn't solve the performance problem. – onerbal Jun 13 '13 at 10:25

score 0 · Answer 4 · answered Jun 13 '13 at 09:51

0

  Session session = sessionFactory.openStatelessSession();
  Transaction tx = session.beginTransaction();
  int i = 0;
  while(it.hasNext()){ 
    i++;
    Object obj = it.next();
    session.save(obj) 
    if (i % 50 == 0) { session.flush(); session.clear();}

statelessSession will help to increase performance.

answered Jun 13 '13 at 09:51

Ruchira Gayan Ranaweera

34,993
17
75
115

Thanks but I forgot to mention that my object has one-to-one relation with another object so in this case using StatelessSession might be a problem. Because StatelessSession does not cascade to composed objects I think. – onerbal Jun 13 '13 at 10:18

Hibernate saveOrUpdate large data

4 Answers4

Linked