2

I found, that method

Iterable<S> CrudRepository#save(Iterable<S> entities)

works very slow.

If instead, I take Iterable myself and create long query like this

INSERT INTO mytable (field1, field2) VALUES
(value1, value2),
(value3, value4),
...
(valueN, valueM);

and execute it against entity manager, it works much faster (by factor of 10 and more).

Can I make Spring-Data itself work the same fast?

For example, are there any options to force Hibernate or any underlying library use such multivalue queries?

I am using SQLite and the class from here: https://stackoverflow.com/a/24233241/258483

May be I can imrpove this class somehow?

UPDATE

I am using identity generation with 

@Entity
@Table(name = "variable_values")
public class VariableValue {

   @Id
   @Column(name="id")
   @GeneratedValue(generator="sqlite")
   @TableGenerator(name="sqlite", table="sqlite_sequence",
      pkColumnName="name", valueColumnName="seq",
      pkColumnValue="variable_values")
   @Getter
   @Setter
   private long id;

does this mean I can't benefit from batching?

Community
  • 1
  • 1
Dims
  • 47,675
  • 117
  • 331
  • 600

1 Answers1

1

You have to specify hibernate.jdbc.batch_size option to enable automatic batching of insert statements. Here are more details of this and related options.

However, you also have to verify if everything is done properly to support it in your custom implementation of the dialect for SQLite.

Community
  • 1
  • 1
Dragan Bozanovic
  • 23,102
  • 5
  • 43
  • 110
  • It is said there, that batching is disabled when identity generating happens. But this is my case. I am not only generating id-s, but do this in strange way suitable for Sqlite! – Dims Apr 28 '17 at 07:41
  • See my update plese. Can I benefit from batching if I use ID generation? – Dims Apr 28 '17 at 07:43
  • Your id generation strategy is not `IDENTITY`, but table-based one, so in theory you can use batching. But as I said in the answer, that may depend on the capabilities of the custom dialect you are using, you'll have to try and investigate it. – Dragan Bozanovic Apr 28 '17 at 13:43