4

I have an application which is running slowly over a WAN - we think the cause is multiple inserts into a table. I'm currently looking into more efficient ways to insert multiple rows at the same time.

I found this method:

INSERT ALL
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (100,20)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (21,2)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (321,10)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (22,13)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (14,121)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (11,112)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (112,23)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (132,2323)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (121,34)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (24333,333)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (1232,3434)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (4554,3434)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (3434,211)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (3434,1233)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (12,22)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (356,233)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (9347,23)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (8904,245)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (342,4545)
  INTO MULTI_INSERT(VAL_1, VAL_2) VALUES (453,233)
SELECT 1 FROM DUAL;

What I would like to know is: is the method above actually more efficient than just doing 20 "INSERT INTO MY_TABLE (1,1);"? Are there other methods of doing this?

ROMANIA_engineer
  • 54,432
  • 29
  • 203
  • 199
user1578653
  • 4,888
  • 16
  • 46
  • 74
  • are there any triggers on the table? – Harshit Aug 06 '13 at 08:51
  • Hi Harshit - No, there are no triggers on the table – user1578653 Aug 06 '13 at 08:56
  • Presumably your real life use case is more than 20 rows? And not hard-coded values? – APC Aug 06 '13 at 09:23
  • @APC, yes that is correct, the table is an audit trail, which inserts roughly 100 rows at a time, all generated by the application at the time, so no hard-coded values. – user1578653 Aug 06 '13 at 09:50
  • Is any checks and unique/primary/foreign keys defined for the table? – ThinkJet Aug 06 '13 at 10:23
  • @ThinkJet No it has no primary key or foreign keys or constraints or indexes - it probably should have, but I didn't design the table! – user1578653 Aug 06 '13 at 10:31
  • 2
    Have you investigated *why* the existing inserts are slow? 100 rows is not a lot, and you should be able to run 100 single insert statements in virtually no time at all. Which suggests that you have some other problem: I/O or CPU contention perhaps, or locking of some description? Rather than guessing at solutions for what may be the wrong problem you should spend some time diagnosis the behaviour of your current code. – APC Aug 06 '13 at 11:53
  • I agree with @APC . Seems that problems arise from table locking. You can try to partition a log table on such a way so the different log sources write to separate partitions. – ThinkJet Aug 06 '13 at 12:41
  • @APC we haven't actually managed to reproduce the problem ourselves - a client reported that it was working perfectly when the application and Oracle were on the same LAN, but when they moved their Oracle servers abroad they say that the program is performing very slowly - this is why I think it may be something to do with sending many inserts over a WAN, and why I'm trying to see if there's a way of doing multiple inserts in one hit. – user1578653 Aug 06 '13 at 12:56
  • @user1578653 Is plain SQL query text with inserts sent over a WAN? – ThinkJet Aug 06 '13 at 13:28
  • @ThinkJet No I just mean that the application is talking to a remote Oracle database over a WAN. – user1578653 Aug 06 '13 at 14:02
  • @user1578653 Ok, is any application server exists between client database and oracle? – ThinkJet Aug 06 '13 at 14:41
  • @ThinkJet There is no application server, it is just a plain Delphi app talking to the database – user1578653 Aug 06 '13 at 15:02
  • @user1578653 So insert query generated in client application or client calls some stored procedure and pass data as parameters? Is client side establish VPN connection to access a database? – ThinkJet Aug 06 '13 at 20:04

4 Answers4

3

You can try direct path insert to speed up operation, but for 100 records conventional path insert must be fast enough and it seems that the problem is about table locking while inserting into log from a big number of sources.

To instruct Oracle to use direct path insert you must specifiy either APPEND or APPEND_VALUES hints depending on insert statement syntax. E.g.

insert /*+ APPEND */ 
into multi_insert(val_1, val_2)
select * from (
  select 100,    20 from dual union all
  select 21,      2 from dual union all
  select 321,    10 from dual union all
  select 22,     13 from dual union all
  select 14,    121 from dual union all
  select 11,    112 from dual union all
  select 112,    23 from dual union all
  select 132,  2323 from dual union all
  select 121,    34 from dual union all
  select 24333, 333 from dual union all
  select 1232, 3434 from dual union all
  select 4554, 3434 from dual union all
  select 3434,  211 from dual union all
  select 3434, 1233 from dual union all
  select 12,     22 from dual union all
  select 356,   233 from dual union all
  select 9347,   23 from dual union all
  select 8904,  245 from dual union all
  select 342,  4545 from dual union all
  select 453,   233 from dual
)

If insert statement originated from PL/SQL code then you can use bulk insert with forall statement to improve performance (SQLFiddle) :

declare
  type TRowList is table of multi_insert%rowtype index by binary_integer;

  vRowList TRowList;
  vRow     multi_insert%rowtype;
begin


  vRow.val_1 := 100;
  vRow.val_2 := 20;
  vRowList(0) := vRow;

  vRow.val_1 := 21;
  vRow.val_2 := 2;
  vRowList(1) := vRow;

  vRow.val_1 := 321;
  vRow.val_2 := 10;
  vRowList(2) := vRow;

  -- ...

  forall vIdx in vRowList.first .. vRowList.last
        insert /*+ APPEND_VALUES */  -- direct path insert
        into multi_insert values vRowList(vIdx);

end;
skaffman
  • 398,947
  • 96
  • 818
  • 769
ThinkJet
  • 6,725
  • 24
  • 33
2

"a client reported that it was working perfectly when the application and Oracle were on the same LAN, but when they moved their Oracle servers abroad they say that the program is performing very slowly"

Okay, so now we're getting somewhere. If you have a set-up in which your hundred statements are individual calls they will probably be sent in separate packets. That would be painful across a WAN compared to a LAN. In that case, it would be worthwhile seeing whether converting statements from RBAR to something Set-based would reduce the number of transmitted packets.

However, I would still advise you to get some hard facts before you roll-out the change. Doesn't your client have a network admin you could talk to? Or at least could you get them to install Wireshark and send you some reports?

APC
  • 144,005
  • 19
  • 170
  • 281
  • We have just managed to set up a remote Oracle server on a VPS and connect our application to it. I did a wireshark capture on it and found that it is sending separate packets for each row. One of my colleagues has recompiled the application, this time using some feature of Direct Oracle Access (the library we use to communicate with Oracle) called "array DML". This sends hardly any packets and is almost instantaneous. However, I would like to know what this "array DML" actually is, as we have other (non-Delphi) apps that could suffer the same problem - can you shed some light on this? – user1578653 Aug 06 '13 at 15:52
  • Probably DOA uses some variation of [APPEND_VALUES hint](http://docs.oracle.com/cd/E11882_01/server.112/e17118/sql_elements006.htm#SQLRF51109) with OCI and binds array parameters to `insert` statement. – ThinkJet Aug 06 '13 at 20:12
  • 1
    AnyDac help have [explanation](http://www.da-soft.com/anydac/docu/frames.html?frmname=topic&frmfile=Array_DML.html) about a topic and [here](http://docs.oracle.com/cd/B28359_01/appdev.111/b28395/oci05bnd.htm#sthref584) is a little bit information from Oracle docs. – ThinkJet Aug 06 '13 at 20:19
0

Some RDBMS like mysql and now SQL Server supports multiple rows insert data syntax:

Insert into myTable ( c1, c2 ) values
( 1,1 ),
( 1,2 ),
... ;

( More details in Inserting multiple rows of data of Sql Server or inserting multirow on mysql )

But don't oracle. Sorry about bad news. The more close way is documented on Tech on the Net.

dani herrera
  • 48,760
  • 8
  • 117
  • 177
0

Oracle doesnt support multi rows insert, please use the next option:

insert into method (name) values ('GET'); insert into method (name) values ('POST');

Jorge Santos Neill
  • 1,635
  • 13
  • 6