Power BI - Get data from post request

Question

I'm trying to get the .cvs file from https://www2.cetip.com.br/TitulosCRI into power BI automatically. In order to do download the file manually, you must first click in the button that says "Enviar" (send, in portuguese) and then in the button that says "Exportar para CSV" (export to CSV).

My first attempt was to find out the download link for the file, but it turned out to be the webpage link itself.

Reading about it, I learnt it might be the case that when I click the buttons, I'm making HTTP post requests. I tried to find some examples of Power BI code to make this kind of request, but my lack of knowledge in the subject made it hard to understand the code provided.

By analyzing the page's source code, I found the code below, which might have something to do with the request:

<input type="submit" name="btExportarCSV" value="Exportar para CSV" id="btExportarCSV" class="button">

Can anyone help me understand how can I get this file automatically using power BI?

Answer 1

Lucca, I believe you're from Brasil, but anyway I'll write in english so people can understand the answer, ok?

I'm trying for some years to scrape this webpage (with R, python and excel+vba), and finally I found a solution: here and here.

I found a solution within Excel + VBA. That I post bellow:

Option Explicit
Public Declare Sub Sleep Lib "kernel32" (ByVal dwMilliseconds As Long)
Private Declare Function FindWindowEx Lib "user32" Alias "FindWindowExA" _
(ByVal hWnd1 As Long, ByVal hWnd2 As Long, ByVal lpsz1 As String, _
ByVal lpsz2 As String) As Long

'------------------------------------------------------

Public Sub AddReference()
    ThisWorkbook.VBProject.References.AddFromFile "C:\Windows\SysWOW64\UIAutomationCore.dll"
End Sub

'-------------------------------------------------------------'
Sub baixa_titulos_CRI()
'Dimensioning the vars
Dim IE As Object
Dim county As String
Dim htmlDoc As Object
Dim sURL As String
Dim buttonclick As Object
Dim ShowMore As Object
Dim exportar_para_CSV As Object
'----------------'
Set IE = CreateObject("internetexplorer.application")
sURL = "https://www2.cetip.com.br/TitulosCRI"
With IE
.Navigate (sURL)
.Visible = True
End With
'I put some waits to run the webpage properly.
WaitIE2 IE, 2000
Set htmlDoc = IE.Document
WaitIE2 IE, 1000
'here it will press the button "enviar"
Set button_send = htmlDoc.getElementById("btEnviar")
button_send.Click
WaitIE2 IE, 2000
'here it will press the button "exportar para CSV"
Set exportar_para_CSV = htmlDoc.getElementById("btExportarCSV")
exportar_para_CSV.Click
WaitIE2 IE, 2000
'this is the solution that I found to press the button "Salvar" in IE.
'I also save the directory that I wanted as default so IE save in the path properly.
'I still couldnt find a solution to name of the file and to the path to save.
'The solution I adapted is to list.files and the date it was saved to compare with older files (as I think you'll do it also I mentioned it here)
Dim o As IUIAutomation
Dim e As IUIAutomationElement
Set o = New CUIAutomation
Dim h As Long
h = IE.Hwnd
If h = 0 Then Exit Sub
Set e = o.ElementFromHandle(ByVal h)
Dim iCnd As IUIAutomationCondition
Set iCnd = o.CreatePropertyCondition(UIA_NamePropertyId, "Save")
Dim Button As IUIAutomationElement
Set Button = e.FindFirst(TreeScope_Subtree, iCnd)
Dim InvokePattern As IUIAutomationInvokePattern
Set InvokePattern = Button.GetCurrentPattern(UIA_InvokePatternId)
InvokePattern.Invoke
WaitIE2 IE, 2000
IE.Quit
Set IE = Nothing
End Sub
'-----------------------------------------------------------
Sub WaitIE2(IE As Object, Optional time As Long = 250)
Dim i As Long
Do
    Sleep time
    Debug.Print CStr(i) & vbTab & "Ready: " & CStr(IE.ReadyState = 4) & _
                vbCrLf & vbTab & "Busy: " & CStr(IE.Busy)
    i = i + 1
Loop Until IE.ReadyState = 4 Or Not IE.Busy
End Sub

I hope it helps

Best, Felipe Ribeiro

[excel] [vba]