0

I am trying to load a file in to database using Ruby. Its a large file about 15 Mb ... it copied the records properly for some time .... but after copying few records, there is no error but it does not insert records in to database ......... and when I connect to Msql prompt in a separate console ... i get an error :

mysql> desc testdb2.test_descriptions;
ERROR 2006 (HY000): MySQL server has gone away                                                                                                                                                                                                                                 
No connection. Trying to reconnect...                                                                                                                                                                                                                                          
Connection id:    52

after this i am able to connect to Mysql database ....... and it is now again the scripts starts writing the records to the database .....

Is there any way to maintain the connection with the database while the app is running ?

I am not sure if its a kind of time out issue or something .... please correct me ....

def simulate_datasets
    Log.initialize

      data_folders = ["Jun_06_2013"];
      data_folders.each do |data_folder|

        add(data_folder);
      end
      render :text => Log.dump
  end

def add (data_folder)
     @dataset  = Dataset.initialize
     @dataset.created_at=Date.new(2013,06,06)
     @dataset.save

     current_root = "script/datasets/"+data_folder+"/"
     strip_string = "/development/A/"
     population_time = {}
     total_time = 0

     clusters = Cluster.find(:all, :order=>"created_at DESC");
     if clusters.empty?
       Log.info "No Clusters found"
       Cluster.initialize
       clusters = Cluster.find(:all, :order=>"created_at DESC");
     end

     clusters.each do |cluster|
        cluster_path = cluster.path
        root = current_root + cluster.name+'/' 
        total_time += populate_file_or_folder(root+"fileListWithMLintMetrics.txt", cluster_path)
     end

end            

I am using populate_file_or_folder method to populate to the database

mysql> show variables like '%time%';
+----------------------------+-------------------+
| Variable_name              | Value             |
+----------------------------+-------------------+
| connect_timeout            | 10                |
| datetime_format            | %Y-%m-%d %H:%i:%s |
| delayed_insert_timeout     | 300               |
| flush_time                 | 0                 |
| innodb_lock_wait_timeout   | 50                |
| innodb_rollback_on_timeout | OFF               |
| interactive_timeout        | 28800             |
| lc_time_names              | en_US             |
| long_query_time            | 10.000000         |
| net_read_timeout           | 30                |
| net_write_timeout          | 60                |
| slave_net_timeout          | 3600              |
| slow_launch_time           | 2                 |
| system_time_zone           | EDT               |
| table_lock_wait_timeout    | 50                |
| time_format                | %H:%i:%s          |
| time_zone                  | SYSTEM            |
| timed_mutexes              | OFF               |
| timestamp                  | 1372869659        |
| wait_timeout               | 28800             |
+----------------------------+-------------------+
20 rows in set (0.00 sec)

def self.populate_file_or_folder(fileName, cluster_path)

    counter = 0                                                      
    # Reading directly from the CSV library
    CSV.foreach(fileName) do |line|
      counter = counter+1
      completePath = line[0]                                                
      completePath = cluster_path+ '/'+completePath

   newStructure = FileOrFolder.new
    newStructure.fullpath = path
    pathbits = path.split('/')
    newStructure.name = pathbits.last
    newStructure.save
    end
  end
Vinay
  • 237
  • 2
  • 8
  • 17
  • I think you are iterating over some set and creating a new mysql connection every time. Make sure you dont create new connections every single time. And please provide more information. Will he helpful for us to understand your problem better. – Anirudhan J Jul 03 '13 at 16:30
  • You might also want to share the results of `mysql> show variables like '%time%';` Some of those might affect what you're doing if they are set horribly wrong (would be odd, but could happen) – Philip Hallstrom Jul 03 '13 at 16:40
  • @PhilipHallstrom ...I have edited my post to include .....the code and results ...pls comment nw .. – Vinay Jul 03 '13 at 16:42
  • @R.A.J edited the post to include the code ... pls check it nw ..Thanks! – Vinay Jul 03 '13 at 16:45
  • @Vinay We need to see `populate_file_or_folder` – Philip Hallstrom Jul 03 '13 at 16:52
  • @PhilipHallstrom included the method populate_file_or_folder in my post .... pls check it nw ... – Vinay Jul 03 '13 at 16:59
  • can we use the reconnect option in the database.yml file and set it to true ..... what changes does it make to the app??? – Vinay Jul 03 '13 at 17:15
  • did u try this http://stackoverflow.com/questions/10474922/error-2006-hy000-mysql-server-has-gone-away | `max_allowed_packet=64M` – AnkitG Jul 03 '13 at 18:28
  • @AnkitG ...how do we set the max_allowed_packet=64M ??? would you like me to set up inside the app or for the mysql in general ?.....I would try this but it seems to me that its because of no activity on the database from the app........what I exactly did was .....i started the rails server, then got an error while running my app(i mean the script to populate data) ...so corrected the code in the app and refreshed the page again... and got this database reconnect issue when i checked it after a while.... – Vinay Jul 03 '13 at 18:47
  • do u think this is the problem ??i should have restarted the server ? because the reconnect is set to false in the database.yml, app is not automatically connecting the database again? I am confused – Vinay Jul 03 '13 at 18:48

0 Answers0