0

Here's my problem:

I get a large csv file with 500k+ rows stored on to my server everyday.

I can access these files through filezilla. What I want to do is to use php to get the latest csv file every day and upload it to my database.

I have tried looping through the rows to add them to the database one by one but that was to no avail. I read online and found that LOAD INFILE might be what i am looking for but I don't know how to implement that to load files that are stored on my server.

Also, I don't know how to automate that upload so it runs every day. Thank you

Isaac Bennetch
  • 11,830
  • 2
  • 32
  • 43
tobel
  • 1
  • 2
  • What issues were you experiencing when you processed the CSV one line at a time? Are the contents of the CSV file actually CSV rows or are they DB `insert` statements? – MonkeyZeus Apr 19 '18 at 15:56
  • You said, :I have tried looping through the rows to add them to the database one by one but that was to no avail". How did you do that? What was the problem that prevented it from working? 500k+ is not that large in the grand scheme of things. – 7 Reeds Apr 19 '18 at 15:57
  • when loop through the file line by line, it only inserts about 50k of the rows. Its not inserting all the rows in my csv files. @MonkeyZeus The contents of the CSV are actual CSV rows not insert statements. – tobel Apr 19 '18 at 18:59
  • Does it produce an error after 50k rows? If yes then what error? – MonkeyZeus Apr 19 '18 at 19:00
  • I figured out a way to Insert all the rows in the CSV by looping through the file and using PHP's fgetcsv() function. Now I want to be able to automate this so it can insert to my database every day. Do you have any idea on how I can go about doing this? – tobel Apr 19 '18 at 19:44

1 Answers1

-1
  • You can check this solution to the same problem that you have.
  • And for the other issue about the scheduled task, you could check tutorials about the timer functions in Javascript

I hope these tips will help you

<button onclick="setTimeout(myFunction, 3000)">Try it</button>

<script>
function myFunction() {
    alert('Hello');
}
</script>
R. García
  • 815
  • 9
  • 20