I am trying to load a file in to database using Ruby. Its a large file about 15 Mb ... it copied the records properly for some time .... but after copying few records, there is no error but it does not insert records in to database ......... and when I connect to Msql prompt in a separate console ... i get an error :
mysql> desc testdb2.test_descriptions;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 52
after this i am able to connect to Mysql database ....... and it is now again the scripts starts writing the records to the database .....
Is there any way to maintain the connection with the database while the app is running ?
I am not sure if its a kind of time out issue or something .... please correct me ....
def simulate_datasets
Log.initialize
data_folders = ["Jun_06_2013"];
data_folders.each do |data_folder|
add(data_folder);
end
render :text => Log.dump
end
def add (data_folder)
@dataset = Dataset.initialize
@dataset.created_at=Date.new(2013,06,06)
@dataset.save
current_root = "script/datasets/"+data_folder+"/"
strip_string = "/development/A/"
population_time = {}
total_time = 0
clusters = Cluster.find(:all, :order=>"created_at DESC");
if clusters.empty?
Log.info "No Clusters found"
Cluster.initialize
clusters = Cluster.find(:all, :order=>"created_at DESC");
end
clusters.each do |cluster|
cluster_path = cluster.path
root = current_root + cluster.name+'/'
total_time += populate_file_or_folder(root+"fileListWithMLintMetrics.txt", cluster_path)
end
end
I am using populate_file_or_folder method to populate to the database
mysql> show variables like '%time%';
+----------------------------+-------------------+
| Variable_name | Value |
+----------------------------+-------------------+
| connect_timeout | 10 |
| datetime_format | %Y-%m-%d %H:%i:%s |
| delayed_insert_timeout | 300 |
| flush_time | 0 |
| innodb_lock_wait_timeout | 50 |
| innodb_rollback_on_timeout | OFF |
| interactive_timeout | 28800 |
| lc_time_names | en_US |
| long_query_time | 10.000000 |
| net_read_timeout | 30 |
| net_write_timeout | 60 |
| slave_net_timeout | 3600 |
| slow_launch_time | 2 |
| system_time_zone | EDT |
| table_lock_wait_timeout | 50 |
| time_format | %H:%i:%s |
| time_zone | SYSTEM |
| timed_mutexes | OFF |
| timestamp | 1372869659 |
| wait_timeout | 28800 |
+----------------------------+-------------------+
20 rows in set (0.00 sec)
def self.populate_file_or_folder(fileName, cluster_path)
counter = 0
# Reading directly from the CSV library
CSV.foreach(fileName) do |line|
counter = counter+1
completePath = line[0]
completePath = cluster_path+ '/'+completePath
newStructure = FileOrFolder.new
newStructure.fullpath = path
pathbits = path.split('/')
newStructure.name = pathbits.last
newStructure.save
end
end