0

I stumbled upon this code but I'm having a hard time getting it working. I'm trying to download a zip file containing a .csv from a website and putting the contents into my excel file. I'm currently stuck at this line:

'3 rename file
Name targetFileCSV As targetFileTXT

It says that it cannot find the file.

Any help is appreciated!

'Main Procedure
Sub LETSDOTHIS()

    Dim url As String
    Dim targetFolder As String, targetFileZip As String, targetFileCSV As String, targetFileTXT As String

    Dim wkbAll As Workbook
    Dim wkbTemp As Workbook
    Dim sDelimiter As String
    Dim newSheet As Worksheet

    url = "http://www20.statcan.gc.ca/tables-tableaux/cansim/csv/00260008-eng.zip"
    targetFolder = Environ("TEMP") & "\" & RandomString(6) & "\"
    MkDir targetFolder
    targetFileZip = targetFolder & "data.zip"
    targetFileCSV = targetFolder & "data.csv"
    targetFileTXT = targetFolder & "data.txt"

    '1 download file
    DownloadFile url, targetFileZip

    '2 extract contents
    Call UnZip(targetFileZip, targetFolder)

    '3 rename file
    Name targetFileCSV As targetFileTXT

    '4 Load data
    Call LoadFile(targetFileTXT)

End Sub

Private Sub DownloadFile(myURL As String, target As String)

    Dim WinHttpReq As Object
    Set WinHttpReq = CreateObject("Microsoft.XMLHTTP")
    WinHttpReq.Open "GET", myURL, False
    WinHttpReq.send

    myURL = WinHttpReq.responseBody
    If WinHttpReq.Status = 200 Then
        Set oStream = CreateObject("ADODB.Stream")
        oStream.Open
        oStream.Type = 1
        oStream.Write WinHttpReq.responseBody
        oStream.SaveToFile target, 2  ' 1 = no overwrite, 2 = overwrite
        oStream.Close
    End If

End Sub


Private Function RandomString(cb As Integer) As String

    Randomize
    Dim rgch As String
    rgch = "abcdefghijklmnopqrstuvwxyz"
    rgch = rgch & UCase(rgch) & "0123456789"

    Dim i As Long
    For i = 1 To cb
        RandomString = RandomString & Mid$(rgch, Int(Rnd() * Len(rgch) + 1), 1)
    Next

End Function

Private Function UnZip(PathToUnzipFileTo As Variant, FileNameToUnzip As Variant)
    ' Unzips a file
    ' Note that the default OverWriteExisting is true unless otherwise specified as False.
    Dim objOApp As Object
    Dim varFileNameFolder As Variant
    varFileNameFolder = PathToUnzipFileTo
    Set objOApp = CreateObject("Shell.Application")
    ' the "24" argument below will supress any dialogs if the file already exist. The file will
    ' be replaced. See http://msdn.microsoft.com/en-us/library/windows/desktop/bb787866(v=vs.85).aspx
     'objOApp.Namespace(FileNameToUnzip).CopyHere objOApp.Namespace(varFileNameFolder).items, 24

'    Call UnZip(targetFolder, targetFileZip)


End Function

Private Sub UnZips(mainFolder As Variant, zipFolder As Variant)


    Call UnZip(targetFolder, targetFileZip)


End Sub


Private Sub LoadFile(file As String)

     Set wkbTemp = Workbooks.Open(Filename:=file, Format:=xlCSV, Delimiter:=";", ReadOnly:=True)

     wkbTemp.Sheets(1).Cells.Copy
     'here you just want to create a new sheet and paste it to that sheet
     Set newSheet = ThisWorkbook.Sheets.Add
     With newSheet
         .Name = wkbTemp.Name
         .PasteSpecial
     End With
     Application.CutCopyMode = False
     wkbTemp.Close

End Sub
Joseph
  • 5,070
  • 1
  • 25
  • 26
RageAgainstheMachine
  • 901
  • 2
  • 11
  • 28

1 Answers1

1

It's because you are extracting the contents of the .zip folder, but the actual filename(s) within that archive is not named data.csv (which is what you're looking to rename, but that file doesn't exist). When I ran the code, the file in that .zip archive was named 00260008-eng.csv.

You need to rename the files that are extracted or look for files that don't have .zip in them after you extract.

Delete this line:

targetFileCSV = targetFolder & "data.csv"

And add a new line within your 1, 2, 3 so you can grab the first CSV file you have from the .zip archive.

'1 download file
DownloadFile url, targetFileZip

'2 extract contents
Call UnZip(targetFileZip, targetFolder)

'3 rename file
targetFileCSV = targetFolder & Dir(targetFolder & "\*.csv")
Name targetFileCSV As targetFileTXT

Also, if anyone else is having trouble running #2 in the code example, add some extra parentheses.

' Added extra parentheses
objOApp.Namespace((FileNameToUnzip)).CopyHere objOApp.Namespace((varFileNameFolder)).items, 24

I have no clue why adding the extra parentheses works, but I couldn't extract the file(s) without it.

Joseph
  • 5,070
  • 1
  • 25
  • 26
  • Do i just need to replace "data" by "00260008-eng.csv". targetFileZip = targetFolder & "00260008-eng.csv.zip" targetFileCSV = targetFolder & "00260008-eng.csv.csv" targetFileTXT = targetFolder & "00260008-eng.csv.txt" I tried this, doesn't seem to work. How do I go about this? Thanks – RageAgainstheMachine Feb 12 '17 at 05:09
  • how do I do this "You need to rename the files that are extracted or look for files that don't have .zip in them after you extract." Thanks – RageAgainstheMachine Feb 12 '17 at 05:18
  • 1
    @RageAgainstheMachine I only replaced `targetFileCSV = targetFolder & "data.csv"` with `targetFileCSV = targetFolder & "00260008-eng.csv"` and it worked for me. However, I wouldn't rely on this method because the name of the file could change with each download. Also, you don't have to rename from `.csv` to `.txt` to open the file in Excel. Excel accepts CSV files. Let me see how I can help with your last comment. – Joseph Feb 12 '17 at 05:20
  • this is awesome, it helps alot! One last thing if you have time...how do I go about having the data go into an existing worksheet, as it stands now it creates a new tab called "data.txt" which makes it challenging if I want to link it to charts and tables. And lastly, how would I go about making this work for different zip files (ex: 00260007-eng.zip, 00260006-eng.zip,00260005-eng.zip) each being printed in their own unique tab? I know this is alot, but any help is very appreciated! Thanks again – RageAgainstheMachine Feb 12 '17 at 05:37
  • 1
    Cool, I'm glad it worked. As for your other questions: #1 You would change the way your `LoadFile()` method works. Instead of adding a new sheet, just reference the sheet you want at the top-left cell to paste in (be careful, though, and make sure you have enough room to paste it in). #2 If the file names won't change from the website, make an array of the file names to download, then in `LETSDOTHIS()` you can loop through the different names and call `DownloadFile()` with different `myURL` variables to download each file. You can create a new temp directory each time you do that to be safe. – Joseph Feb 12 '17 at 05:46
  • 1
    @RageAgainstheMachine Also, if this has been helpful, could you please give it an upvote and accept the answer? – Joseph Feb 12 '17 at 05:48
  • How would I go about downloading only a portion of the data instead of the full data set? – RageAgainstheMachine Jun 24 '17 at 00:48
  • @RageAgainstheMachine if you're calling a URL that has the `.zip` file extension, you are going to attempt the download the entire file. You could _technically_ download a portion of the data, but you won't be certain if you will be able to unzip only a portion of the data, and chances are extremely likely that the file would be corrupt, so it's just not worth it. What are you planning to do, exactly? – Joseph Jun 24 '17 at 14:33
  • Thanks for the response, I was trying to do this to reduce the size of the file...it's getting heavy. – RageAgainstheMachine Jun 24 '17 at 14:41
  • @RageAgainstheMachine how often do you need to do this? Is it automated? Or do you run it when needed? And how often does the data get updated on the server? – Joseph Jun 24 '17 at 17:17
  • @josephftw this will be used once a month, its automated, data gets updated monthly – RageAgainstheMachine Jun 25 '17 at 02:02
  • @RageAgainstheMachine I wonder what makes this heavy for you, is it the file size or the processing of the data as it grows? If it's the latter, and the file is the same file name every time you download it. Then as long as it always grows and is never corrected you can have a local file save your last position in the file, then the next time you process it you can open the file and begin processing where you left off. I would make this another question on SO, though. Feel free to leave the link in a comment if you do. – Joseph Jun 25 '17 at 13:06
  • @josephftw For some reason, lately when I've been running it I've been getting the message "out of memory", I'm not sure how to address this. Also, I've been linking all the data to other sheets and graphs using VLookUp which appears to be slowing this down significantly, would you know how to get around this? Thanks for all the help! – RageAgainstheMachine Jul 14 '17 at 13:49
  • @RageAgainstheMachine sounds like the computer it's running on is probably running out of ram when it's executing. Most-likely when you're copying/pasting the data. However, the file size is so small that I doubt the code you show in the original question is at fault. I would recommend creating another question for this. Also, see https://stackoverflow.com/questions/14396998/how-to-clear-memory-to-prevent-out-of-memory-error-in-excel-vba – Joseph Jul 15 '17 at 19:42