1

I'm trying to read from a mySQL server (in my LAN) a huge resultSet. I implemented it the way I found looking for the setFetchSize method: BalusC's Example. So I did:

        con = DriverManager.getConnection(url, user, password);

        //con.setAutoCommit(false);
        st = con.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
        st.setFetchSize(Integer.MIN_VALUE);
        rs = st.executeQuery(query);

        int count = 0;
        while (rs.next ()) {                
            // Process entry
            String rsc = rs.getString ("resource");
            String tpc = rs.getString("topic");   
            System.out.println(count + ": " + rsc); 
            ++count;                
        }

Although, it hangs at row 1077 at the line rs.next(). It doesn't throw exception.

EDIT:

After a timeout something new happened. It returned this error message:

JDWP exit error JVMTI_ERROR_NONE(0): getting frame location [../../../src/share/back/stepControl.c:641]

This error seems to be unrelated.

EDIT2: I've coded a PHP script for retrieving results by stream and happens exactly the same. So it has nothing to do with JDBC... I don't know what is going on!!

Community
  • 1
  • 1
synack
  • 1,699
  • 3
  • 24
  • 50
  • `Integer.MIN_VALUE` is a negative value. Only 0 and positive values are defined in `setFetchSize()`. What goal did you want to achieve with this call? – Joachim Sauer May 16 '12 at 10:35
  • No, as I've found is the way to do it for JDBC mySQL – synack May 16 '12 at 10:35
  • 1
    For example: http://stackoverflow.com/questions/2180835/handling-large-records-in-a-j2ee-application Btw, @BalusC seems an expert on this topic. – synack May 16 '12 at 10:37
  • 3
    Oh, the MySQL folks have butchered the standard again. How *nice* of them. I'm afraid I can't help you here. – Joachim Sauer May 16 '12 at 10:38
  • 1
    Interesting, so a negative size means "one row at a time". I wonder why *setFetchSize(1)* wasn't chosen. Also conflicts with the PAI doc that says setFetchSize throws exception for values < 0. – Roger Lindsjö May 16 '12 at 13:17

2 Answers2

2

The second error message is a deadlock in the Eclipse debugger when hot-swapping and/or JRebel (source):

It is a known problem, actually two problems.
1) Eclipse debugger deadlocks sometimes on hot-swapping (true for any version of JRebel).
2) JVM crash on hotswapping - it is true for JRebel 4M1.

We are expecting bugfix release soon (3.6.2 together with Eclipse plugin) - It will fix first problem. Second problem should be fixed with 4M2 release.

Suggestions:

  1. Update Eclipse to 3.7.2 and JRebel to the latest release
  2. Start the application again.
  3. Use logging/System.out.println() to debug
  4. Check the log of your mysql server for problems (harddisk, network)
  5. Run the query in a SQL workbench (to see whether this is a problem in the query/the server or your code)
Aaron Digulla
  • 321,842
  • 108
  • 597
  • 820
  • Thank you for your response. I also found this information when looking for a solution in google. 1. I don't use JRebel and use the last version of Eclipse. 2. I've already done that, even executing the application on the same machine where the mySQL server is located. 3. I've found at what line it hangs... What more information can provide that? 4. This is the only thing that I haven't done yet. I'll try and tell you something. 5. Done. I tried executing the query in the mySQL propmt and I retrieved 1080 rows correctly. – synack May 16 '12 at 17:34
  • I checked the /var/log/mysql.err and /var/log/mysql.log and they are empty! – synack May 18 '12 at 07:17
  • Does it hang at the first row or can it read some and hangs then? And can you check whether it's a GC issue? See here how to enable GC logging: http://christiansons.net/mike/blog/2008/12/java-garbage-collection-logging/ – Aaron Digulla May 18 '12 at 07:58
  • finally I found a solution, I'll post it in few minutes. – synack May 18 '12 at 10:28
1

OKay, the problem was at the query itself.

I've found that streaming is very sensitive on how query is built. If I make a little change (like adding SELECT DISTINCT) it buffers or hangs. Finally, I left a column out of the query and worked...

synack
  • 1,699
  • 3
  • 24
  • 50