11

Possible Duplicate:
Array size too big - ruby

Sorry if this has been asked, I looked around but didnt really find what I was looking for.

I am using ruby and mysql to create an array based off of a single column in the mysql table. So for example, say I have a column of user names:

 users = []
 users.clear
 
 # Update the list of users to follow
 res = dbh.query("SELECT user FROM usernameDB")
 while row = res.fetch_row do
   users << row[0] #adds each user to the array
 end

This has worked fine up until now, when we started recieving a lot more users. Now the code gives me unknown errors.

In an attempt to troubleshoot, I simply commented out most of it and built the array with just a couple of user names, and everything worked again. So my question is, is there a limit to the number or items in a ruby array?

Thanks!

bradleygriffith
  • 948
  • 3
  • 11
  • 24
  • 2
    Can you report the 'unknown errors'. – knut Jul 28 '11 at 20:38
  • I have a similar issue. I am building up an array with data from an ActiveRecord query, and at certain large sizes it fails or times out. Like others have mentioned, it's not that the array is too large - there must be something else going on. The total length of my array should end up in the thousands, not millions.... – Drew Nov 13 '15 at 17:41

2 Answers2

7

There is not a software limit imposed by Ruby, but there is a limit as to how much the process can support. If you have a regular home server running the Ruby server, it would be able to handle an array until the array became too large, at which point it would begin to 'bog down', lag, crash, etc. On the other hand, if you had an extremely powerful corporate server, it could handle a much larger array, but would still eventually crash/lag if the array became too large for the process and the hardware (memory) to handle.

I don't have any concrete numbers for you, because it all depends on the hardware and software on the server.

Brad Larson
  • 170,088
  • 45
  • 397
  • 571
dwmcc
  • 1,034
  • 8
  • 19
  • It's running on dreamhost. Sorry if that is the wrong answer, I'm a bit of a nub to ruby in general. – bradleygriffith Jul 28 '11 at 20:31
  • No problem. It really depends what resources Dreamhost has at their disposal for their servers. I think you could build a generous size array, but anything that encompasses many many items may be too much. Just guess and check - and if you get an email from Dreamhost asking where their server resources are going, don't name me! :) – dwmcc Jul 28 '11 at 20:32
4

Take a look at this other post: Array size too big - ruby. The size of 600 million was too big, but 500 million worked.

How big of an array are you working with? The problem may be that you're running out of memory!

Community
  • 1
  • 1
danmcardle
  • 2,079
  • 2
  • 17
  • 25
  • there are 436 rows in the table so Im trying to build an array of 436 usernames. – bradleygriffith Jul 28 '11 at 20:26
  • 4
    436 usernames is nothing. There must be someting else going on. – Mchl Jul 28 '11 at 20:32
  • That doesn't sound so bad. Does it still come crashing down if you limit the number of results returned by the query to 50 or so with `SELECT TOP 50`? – danmcardle Jul 28 '11 at 20:32
  • I started playing around with limits. At 100 it worked fine, at 200 it worked fine, at 400 it errored out again so I set it to 350 and it worked fine. After some narrowing down, it seems it works fine up until I set the limit to 354. I checked that entry in the database and it appears fine. – bradleygriffith Jul 28 '11 at 20:42
  • 1
    Well, for sure, this isn't due to some constraint ruby imposes on the length of arrays. – danmcardle Jul 28 '11 at 20:47