32

When I search the web for inserting BLOBs into Oracle database with jdbc thin driver, most of the webpages suggest a 3-step approach:

  1. insert empty_blob() value.
  2. select the row with for update.
  3. insert the real value.

This works fine for me, here is an example:

Connection oracleConnection = ...

byte[] testArray = ...

PreparedStatement ps = oracleConnection.prepareStatement(
    "insert into test (id, blobfield) values(?, empty_blob())");
ps.setInt(1, 100);
ps.executeUpdate();
ps.close();
ps = oracleConnection.prepareStatement(
    "select blobfield from test where id = ? for update");
ps.setInt(1, 100);
OracleResultSet rs = (OracleResultSet) ps.executeQuery();
if (rs.next()) {
    BLOB blob = (BLOB) rs.getBLOB(1);
    OutputStream outputStream = blob.setBinaryStream(0L);
    InputStream inputStream = new ByteArrayInputStream(testArray);
    byte[] buffer = new byte[blob.getBufferSize()];
    int byteread = 0;
    while ((byteread = inputStream.read(buffer)) != -1) {
        outputStream.write(buffer, 0, byteread);
    }
    outputStream.close();
    inputStream.close();
}

There are some webpages where the authors suggest using a simpler 1-step solution. Previous example with this solution:

Connection oracleConnection = ...

byte[] testArray = ...

PreparedStatement ps = oracleConnection.prepareStatement(
    "insert into test(id, blobfield) values(?, ?)");
BLOB blob = BLOB.createTemporary(oracleConnection, false, BLOB.DURATION_SESSION);
OutputStream outputStream = blob.setBinaryStream(0L);
InputStream inputStream = new ByteArrayInputStream(testArray);
byte[] buffer = new byte[blob.getBufferSize()];
int byteread = 0;
while ((byteread = inputStream.read(buffer)) != -1) {
    outputStream.write(buffer, 0, byteread);
}
outputStream.close();
inputStream.close();

ps.setInt(1, 100);
ps.setBlob(2, blob);
ps.executeUpdate();
ps.close();

The second code is much more easier, so my question is: What is the point of first (popular) solution? Is there (was there) some kind of constraint for the second solution (Oracle server version number, jdbc driver version, size of the blob,...)? Is the first solution better (speed, memory consumption,...)? Any reasons for not using the simpler second approach?

The exact same question applies for CLOB fields.

asalamon74
  • 6,120
  • 9
  • 46
  • 60

9 Answers9

11

The update approach you mention in the first case can be rewritten using pure JDBC code and thus reduce your dependency on Oracle-specific classes. This could be helpful if your app needs to be database agnostic.

public static void updateBlobColumn(Connection con, String table, String blobColumn, byte[] inputBytes, String idColumn, Long id) throws SQLException {
  PreparedStatement pStmt = null;
  ResultSet rs = null;
  try {
    String sql = 
      " SELECT " + blobColumn + 
      " FROM " + table + 
      " WHERE " + idColumn + " = ? " +
      " FOR UPDATE";
    pStmt = con.prepareStatement(sql, 
      ResultSet.TYPE_FORWARD_ONLY, 
      ResultSet.CONCUR_UPDATABLE);
    pStmt.setLong(1, id);
    rs = pStmt.executeQuery();
    if (rs.next()) {
      Blob blob = rs.getBlob(blobColumn);
      blob.truncate(0);
      blob.setBytes(1, inputBytes);
      rs.updateBlob(blobColumn, blob);
      rs.updateRow();
    }
  }
  finally {
    if(rs != null) rs.close();
    if(pStmt != null) pStmt.close();
  }
}

For MSSQL I understand that the locking syntax is different:

String sql = 
  " SELECT " + blobColumn + 
  " FROM " + table + " WITH (rowlock, updlock) " + 
  " WHERE " + idColumn + " = ? "
Mr. Shiny and New 安宇
  • 13,822
  • 6
  • 44
  • 64
  • 1
    Is the "FOR UPDATE" clause database agnostic? It doesn't seem to work in SQL Server, for example. But maybe it's not essential to the above technique anyway... – Andrew Swan Jun 24 '09 at 04:50
  • The FOR UPDATE clause instructs the database to lock the rows. It's important in this case because the resultset is going to update those rows. If you are reasonably certain that nobody else will update those rows, you might not need the lock. – Mr. Shiny and New 安宇 Jun 24 '09 at 12:35
  • 5
    You are not inserting a BLOB here, you are just updating it. The OP was specifically about inserting. – Dan Berindei Mar 25 '11 at 17:56
7

Another point of view from Oracle DBA. Sun guys did very poor job when they designed JDBC standards(1.0, 2.0, 3.0, 4.0). BLOB stands for large object and therefore it can be very large. It is something that can not be stored in JVM heap. Oracle thinks of BLOBs as something like file handles(it fact they are call then "lob locators"). LOBS can not be created via constructor and are not Java objects. Also LOB locators(oracle.sql.BLOB) can not be created via constructor - they MUST be created in the DB side. In Oracle there are two ways how to create a LOB.

  1. DBMS_LOB.CREATETEMPORATY - the returned locator in this case points into temporary tablespace. All the writes/reads against this locator will be sent via network onto DB server. Nothing is stored in JVM heap.

  2. Call to EMPTY_BLOB function. INSERT INTO T1(NAME, FILE) VALUES("a.avi", EMPTY_BLOB()) RETURNING FILE INTO ?; In this case returned lob locator points into data tablespace. All the writes/reads against this locator will be sent via network onto DB server. All the writes are "guarded" by writes into redo-logs. Nothing is stored in JVM heap. The returning clause was not supported by JDBC standards (1.0, 2.0), therefore you can find many examples on the internet where people recommend approach of two steps: "INSERT...; SELECT ... FOR UPDATE;"

Oracle lobs must be associated with some database connection, they can not be used when DB connection is lost/closed/(or "commited"). They can not be passed from one connection to another.

You second example can work, but will require excessive copying if data from temporary tablespace into data tablespace.

Ivan
  • 71
  • 1
  • 1
  • So in your last case how would java code be written? How should "RETURNING FILE INTO ?" be handled? – Askar Kalykov Sep 25 '14 at 14:08
  • I don't understand this answer fully in terms of what JDBC 4.0 actually allows, such as having a prepared statement such as: stmt = conn.prepareStatement("insert into storage (storage.storageid,storage.attachment) values (?, ?)"); ... stmt.setBlob(1, fInputStream); ... What exactly do you mean when you say in Oracle there are only those two ways to create Blobs, since above statement works on Oracle (with ojdbc6.jar driver)? – ammianus Sep 29 '14 at 16:32
  • 4
    OMG - I just want to insert a few lines of text from a String into a "BLOB" in a schema I inherited. The String isn't MBs, let alone GBs, of data, BUT, it is sometimes over the astounding limit of 2000 characters. What a circus. – Roboprog Jan 09 '16 at 00:05
5

The Oracle server's LOB handling is pretty poor and can suffer from serious performance problems (e.g. massive overuse of the redo log), so the first solution may be a way to address those.

I would suggest trying both approaches. if you have a competent DBA, they may be able to advise which approach has the lowest impact on the server.

skaffman
  • 398,947
  • 96
  • 818
  • 769
5

One interesting thing with JDBC is you can upgrade rather aggressively to the latest drivers and work with JDBC 4.0 features. The oracle JDBC drivers will work with older database versions, so you can use an 11g branded JDBC driver against a 10g database. The Oracle database 11g JDBC comes in two flavors: ojdbc5.jar for Java 5 (i.e., JDK 1.5) and ojdbc6.jar for Java 6 (i.e., JDK 1.6). The ojdbc6.jar supports the new JDBC 4.0 specification.

With the newer drivers/jdbc 4.0 you can create Blobs and Clobs off the connection object:

Blob aBlob = con.createBlob();
int numWritten = aBlob.setBytes(1, val);
Brian
  • 13,412
  • 10
  • 56
  • 82
  • 3
    Unfortunately Oracle did a poor job implementing the standard JDBC way of BLOB handling (maybe for business reasons). You are forced to work with concrete oracle classes to get the job done properly. – akarnokd Jun 24 '09 at 12:42
  • It doesn't seem to work for me: JEE container and Oracle 10.2. – Luis Soeiro Jun 06 '11 at 18:33
  • @LuisSoeiro - Verify what JDBC driver your using. – Brian Jul 09 '12 at 17:44
4

This statement :

blob.setBytes(1, inputBytes);

is giving issues when I use oracle thin client ojdbc14.jar, "Unsupported Features"

So, I had to work around by :

rset.next();
Blob bobj = rset.getBlob(1);
BLOB object = (BLOB) bobj;
int chunkSize = object.getChunkSize();
byte[] binaryBuffer = new byte[chunkSize];
int position = 1;
int bytesRead = 0;
int bytesWritten = 0, totbytesRead = 0, totbytesWritten = 0;
InputStream is = fileItem.getInputStream();
while ((bytesRead = is.read(binaryBuffer)) != -1) {
bytesWritten = object.putBytes(position, binaryBuffer, bytesRead);
position += bytesRead;
totbytesRead += bytesRead;
totbytesWritten += bytesWritten;
is.close();
sra
  • 23,820
  • 7
  • 55
  • 89
Yinch
  • 1
  • 1
2

Some watchouts found for the second solution

I am using ojdbc6.jar - the latest release and for the statement from 'the second solution':

BLOB blob = BLOB.createTemporary(oracleConnection, false, BLOB.DURATION_SESSION);

I have to release blob after the statement is completed - or otherwise blob is closed when session is closed (which can take long time with connection pooling).

blob.freeTemporary();

Otherwise you can see locked resources:

select * from v$temporary_lobs

Another problem with temporary BLOBs is the need to allocate temporary tablespace: as per documentation http://docs.oracle.com/cd/E11882_01/appdev.112/e18294.pdf

Managing Temporary Tablespace for Temporary LOBs Temporary tablespace is used to store temporary LOB data

j23
  • 160
  • 9
2

Provided the CLOB data is small enough to fit in your memory without blowing up, you can just create a prepared statement and simply call

ps.setString(1, yourString);

There may be other size limitations, but it seems to work for the sizes we're dealing with (500kB max).

Quartz
  • 1,731
  • 1
  • 14
  • 17
1

I found a simple call to setObject(pos, byte[]) works for my case. From Database Programming with JDBC and Java By George Reese,

        byte[] data = null;
        stmt = con.prepareStatement("INSERT INTO BlobTest(fileName, "
            + "blobData) VALUES(?, ?)");
        stmt.setString(1, "some-file.txt");
        stmt.setObject(2, data, Types.BLOB);
        stmt.executeUpdate();
Kirby
  • 15,127
  • 10
  • 89
  • 104
  • 1
    It is probably a good idea to force the type by adding [java.sql.Types.BLOB](http://docs.oracle.com/javase/7/docs/api/java/sql/Types.html#BLOB) to the list of arguments of `setObject`. – ceving Jun 05 '13 at 08:00
  • Thank you, @ceving. I learned something. :) I updated the answer – Kirby Jun 05 '13 at 15:25
0

If size of inserting BLOB is greater than blob.getBufferSize(), transaction is commited as soon as first chunk is written to db as default value of autoCommit property of jdbc connection is true and further chunks writes fail as db treats them as new transactions. It is suggested as follows:
a) Set jdbc connection autoCommit property to false.

conn.setAutoCommit(false);

b) Explicitely commit the transaction after uploading the whole BLOB.

while ((bytesRead = messageInputStream.read(buffer)) != -1) {
     cumBytes += bytesRead;
     blobOutputStream.write(buffer, 0, bytesRead);
    }
conn.commit();
AVA
  • 2,474
  • 2
  • 26
  • 41